Practical Face Reconstruction via Differentiable Ray Tracing

We present a differentiable ray-tracing based novel face reconstruction approach where scene attributes – 3D geometry, reflectance (diffuse, specular and roughness), pose, camera parameters, illumination are estimated from unconstrained monocular images. The proposed method models via novel, parameterized virtual light stage, which in-conjunction with ray-tracing, introduces coarse-to-fine optimization formulation for reconstruction. Our can not only handle self-shadows conditions, but also estimates diffuse albedos. To estimate the consistently practical semantics, two-stage strategy systematically uses subset of parametric attributes, subsequent attribute estimations factor those previously estimated. For example, during first later prevent its baking into personalized albedos in second stage. show efficacy our several real-world scenarios, be even under extreme conditions. Ablation studies, analyses comparisons against recent state-of-the-art methods improved accuracy versatility approach. With consistent reconstruction, leads to style illumination, albedo, self-shadow edit transfer applications, as discussed paper.